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llna a«M«r dsarr l tier. a hand'touch sensing nystci* that , when mounted on a pos 1 1 1 or^r— -■ -» 
controlled manipulator, ca"n greatly expand the klnda of automated manipulation tank* that 
can be undertaken. Beeauae of the variety of coordinate conversion*, control equation*, 
and completion criteria, control la necessarily dependent upon a small digital computer. 

The sensing system Is designed both to be rugged and to sense the necessary touch und force 
Information required to execute a wide range of manipulation taskr.. The system consists of 
a six-axis wrist .ensor, external touch sensors, and a pair of matrix Jaw sensors. Details 
of the construction of the particular sensors, the Integration of th'j end effector Into the 
sensor system, and the contrr’ algorithms for using the sensor ou'.put* to perform manipula- 
tion tnsks automat lea 1 lv are discussed. 


INTRODUCTION 

Current Industrial robots are devices that move 
from position to position under preprogrammed control. 
Semmerllng (1972) describes them as follows 

... easily programmable, operatorless handling 
devices that can perform simple, repetitive 
lob* .hat require few alternative actions and 
ml.ilmum communication with the work environ- 
ment. They are unable to think, see, hear, 
smell, or taste, and only In some instances 
can they be given a rudimentary sense of feel. 

Whenever there are sufficient variations In the posi- 
tions of objects to be picked up or motion constraints 
on an object to be moved, the conventional, (.osttton- 
controlled manipulator cannot carry out the task. Re- 
search at SRI and other laboratories In the United 
States and in Japan has begun to show now touch a.id 
force sensing In robots, together with the proper con- 
trol system (usually based on a small computer), ran 
be used to solve these problems and to make robots 
more useful . 

Table 1 lists several areas In which touch 
sensing can be used to expand the range of manipula- 
tion tasks. Each of these uses requires particular 
touch sensors und a particular control algorithm Tor 
accomplishing the task. Thus, in designing a touch 
sensing system for automatic manipulation, both the 
quantity to be sensed and the type of control al- 
gorlthi liable must lie considered. The sensing 

system d ici 1 bed in this paper Includes sensors that 
can be used in all of the tasks In Table I, 


This work was supported by the National Aeronautics 
and .Space Administration, untlec. Cont racts SNSN-63 
and NAS 2-7507 , and by the National S< lence inunda- 
tion, under Grant Cl -38100x. 


USES OK TOUCH INFORMATION 

Correcting position errors 
•Bringing mating parts together 

• Starting pins Into hole* 

• Locating surfaces, turners, edges, and the like 
Acqul si t ion 

• Aligning Jaws to objects 

• Extracting one part from a bln of pnrts 

Constrained motion 

• Sliding pa rt s 

• Final insertion of pins into holes 

• Turning cranks, or hinged doors 

Error detection 

• Col 1 is ions 

• Acquisition failures 

• Task completion [allures 

Training (or programming) the manipulator by 
pushing on hand 

• Steering through tasks 

• Setting force levels 

Classification of objects 

• Size 

• Weight ( 

• Shape I 

• Motion constraints 


TOUCH CONTROLLED MANIPULATION 

To assemble parts, information from touch sensors 
ran lie used to steer the hand ns It closes and moves. 

A simple e . ample of this procedure is that of aligning 
the hand to an object without disturbing It, a* Illus- 
trated In Figure 1. This alignment procedure may be 
required either to pick up an object without knocking 
it over or to calibrate the hand to part of any object 
for subsequent mating of parts to that object. For 
such purposes, sufficiently sensitive sc-sors are 
needed on the gripping surfaces of the fingers to de- 
tect finger contact with an object without pushing it 
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Paper pretented at tha 17th Annual Human Factor* Society Convention Wethmr in. D.C., October 16 18, 1973. 


Coordinate System* 
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FIGURE 1 ALIGNING FINGERS TO OBJECT 
Jaw dOMt («) until light touching 
contact it mad# <b) Than antira 
hand it moved at jaw doting tpead 
until both tongt contact objacf 4c). 

\s the Improperly centered f inner* shown tn Fig- 
ure 1(a) close on the object, contact against one 
f inner t* made. The computer control system must 
then cause the* hand to sweep In a direction from one 
finger to another, in a coordinate system determined 
by the hand, while the f inner* continue to close 
[Figure 1(b)]. Closing am* sweeping proceed until 
both fingers contact the obtect, as shown in Figure 
1(c). At this point, the control system must termi- 
nate the grasping process and activate the next step 
in the assembly algorithm. This example shows that 
several separate abilities are required for success- 
ful manipulation based on touch control 

• Motion in different coordinate systems. 

• The ability to steer the hand relying on 
touch . 

• Determination of critical forces for carry- 
ing out the task. 

• Determination of task completion criteria 
based on touch. 

• The means for measuring these critical 
forces . 

In the following four sections, the implications of 
the above requ i rements are briefly discussed, and 
their importance to the design of a general purpose 
end effector with a built-in touch sensing system is 
descr i bed . 


Before describing bow sensors are used to control 
the manipulator, it is necessary to define the coordi- 
nate systems in which the manipulator must move. Any 
manipulator is controlled in an arm root .1 mate system 
that is uniquely determined by its own geometry; there 
are as many coordinates as there are movable Joints in 
the manipulator. Arm coord i na tes , however, are of 
little use in the automatic manipulation tusks of in- 
terest here. To assemble parts, it is necessary to 
move the manipulator holding the daughter part in the 
coordinate system of the mother part. On the* other 
hand, when maneuvering In the working area, it Is nec- 
essary to move in the coordinate system of the work 
space. This is particularly useful when maintaining 
the hand at a certain height above the* floor and ta- 
bles and still being able to slide objects across 
the*m. liy placing parts on a motorl /ed turntuble, and 
by using Jigs and fixtures, it is possible to cause 
the coordinate system of the mother part to coincide 
with that of the work space, thus simplifying the ma- 
nipulator control problem. Similarly, by either care- 
fully designing the end effector to mute with the 
daughter part or by designing Jigs to hold or align 
the part as it is being picked up, the coordinate sys- 
tems can be fixed with respect to one another, again 
simplifying the control equation*. 

The two most important coordinate systems in 
which the arm must be able to move for automatic- 
controlled ussembly operations are therefore work- 
space coordinates and hand coordinates. Hiese are il- 
lustrated in Figure 2. The mathematics for moving a 
manipulator in these coordinate systems for particular 
applications has been discussed by both Whitney (1*169) 
and Paul (1972). 

Control Algorithms 


To perform useful tusks, the information f rom 
touch sensors must 1m* used to control the position of 
the manipulators. When the hand is close to the area 
of the oh |ect to oe picked up, the mot ion of the hand 
must be* ‘eered by the actuation of sensors so that 
(1) the oh feet will not be knocked about and (2) a 
secure grip wlil be maintained. 

The situation can be compured to the hypothetical 
requirement that u yardman in a railroad switchyard 
walk up to u inn- ton engine and push It along the 
truck with his bare hands. The problem can be solved 
simply by installing the throttle (u proportional 
touch sensor) on the front of the engine within reach 
of the yardman. Ily exerting a pound or so of force on 
the throttle, he can then move the 100- ton engine. 

The* harder he pushes, the faster the engine will go. 

Similarly, the "power steering required for the 
self-centering grip shown in Figure 1 causes the hand 
to H**»ep left or right, depending on whether the lett 
or right gripping surface of the finger Is pushed. 

The harder the push, the faster the hand should sweep. 
To accomplish this task, the control algorithm must 
move the Joints of the manipulator In a particular 
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coordinated fashion In response to the proportional 
inputs from touch sensors on the Inside surface of 
the (aw. Like power steerinit. a small force will 
cause an otherwise immobile manipulator to move 
freely. 

A second example of an acquisition strategy Il- 
lustrating a particularly desirable combination of 
sensor-directed motions Is shown In figure 3. After 
sweeping, the hnnd Is directed to move about the turn 
and tilt axes by signals from touch sensors on the 
gripping surfaces of the Jaws This strategy is use- 
ful for acquiring objects without moving them or for 
determining the position, sire, and orientation of an 
Imprecisely known object. The task requires sensing 
both small, proportional torques used to drive the 
turn and tilt axes ard the light proportional pres- 
sure developed on the Inside surface of the tongs 
used to drive the sweep axis. The closing of the 
hand generates these forces, and task completion Is 
indicated by the attainment of some threshold grip- 
ping force. For this task, the most appropriate lo- 
cation for sensors is on the Inside surfaces of the 
Jaws. 


FIGURE 3 ROTARY ALIGNMENT TO OBJECT 
At the |awn clot*, the hand it driven 
tint to turn (a) and then to till Ibl 
by tignalt derived from proportional 
force teuton on |awr turfacet to 
achieve detired orientation (cl for 
gratping 

and the proper alignment of hand coordinates to block 
coordinates using the previous acquisition strategy. 
The first step in the placement task [Figure 1(a)] In- 
volves the assumption of the parent-part coordinates 
by the end effector. This Is done by allowing the 
hand to tilt and turn to nullify torques that build 
up ns the block Is lowered to and pressed against the 
parent surface. When a threshold reach force builds 
up, the first portion of the task Is complete, and the 
hand must then be controlled to lift, to maintain 
reach pressure, and to nullify twist torque. This 
brings the second block fnee to mate with the second 
parent surfaco [Figure 4(b)], When a threshold lift 
force Is obtained [Figure 4(c)], the task is complete. 
The Jaws are then opened while holding the hind In Its 
posl t Ion. 


A different example, a placement task. Is illus- 
trated In Figure 4. Here the task Is to push a block 
Into a mating corner. The control problem is simpli- 
fied both by the proper alignment of the coordinates 
of the mother part with the work-space coordinates 


Control Equations 


Control of the manipulator to aasume various pos 1 - 
tinns, to move at different rates, and to apply forces, 
Is accomplished by selecting and Implementing the 








I .GURE 4 AUTOMATIC PLACEMENT OF BLOCK 

Block it placid Hat aptmtt a turfaca (a) 
by lowering it until contact forca it 
matur'd at writ! and than rotating hand 
on two a rat to null mitalignmant torquet 
At downward prattura it maintained 
(light arrow in bl on block, it it tlid latt 
(dark arrow in bl until tufficient contact 
force it built up (cl. 

proper control equation for each of the coordinate 
axes. The well-known equation for position la 


Hate * K (P - P) , (1) 

,i c 

where la the poaltton Kiiln, P ( , i* the commanded 
poaltion, and I’ la the actual position. For control 
In hand coordinates, Kate, P c and P can be considered 
to be 1 x 7 matrices that spei I fy the corresponding 
rates or positions of the seven hand coord -lates. To 
obtain altdtntc alone a particular axis, the control 
equation la more simply expressed as 

Kate . K (2) 

c 


where R c la the command rate matrix. To control force, 
the general force generating equation la 

Kate ■ K (F • F> . (.1) 

f c 


If K ( , the force gain, Is zero, the hand la stiff nnd 
will not respond to external forces If K f Is large, 
then the hand moves quickly to generate or respond to 
external forces. If the command force vector, f r , la 
zero, the hi id moves freely wherever It la pushed. If 
F c la not zero, the hand moves until forces ate devel- 
oped on particular force sensors (F) that mutch t, . 
There can be one or many more than seven force sensors. 

It la useful to combine Eqs. (I), (2), and (3) 

Into the general control equation given below. 


Rate W K (P - P) ♦ R ♦ K (F - F) . (I) 

pc c f c 


By properly choosing the gams In Eq. ( I) , the hand 
can be made to perform the following actions simulta- 
neously • 

• To push on one axis. 

• To move on another at a fixed rate. 

• To hold a third fixed. 

• To make the remaining four axes passive 
to external forces or torques. 

Performing the sequences of tasks previously shown 
In Figures 3 and ■! requires (t) a sequence of different 
control equations based on Eq. (I), and (2) propor- 
tional sensors to measure those forces pertinent to the 

task . 

Completion Criteria 

To determine when the transition from one set of 
control equations to another should be made, completion 
criteria must be established nnd continuously tested. 
Some examples of these criteria, based on force sens- 
ing, are given In the previous tasks (Figures 1, 3, and 
■1), In general, many different completion conditions 
must be specified during any manipulation task. 

Equally important to subtask completion are those cri- 
teria that indicate improper operation of the system. 
Examples of both kinds of criteria are given In Table 2. 

With each control equation. It Is necessary to 
specify both a list of completion criteria and the new 
actions and control equations to be used If any of 
these criteria are met. This suggests that a branching 
structure associated with a computer language Is re- 
qi i rod to specify both the manipulation task and any 
required emergency procedures. These procedures should 
cause the hand to stop In midtask and should Inform the 
human supervisor of any difficulties and their symptoms. 



Table 2 

EXAMP LB OE COMPLETION CRITERIA 


Workspace coordinates 

• Exceeded work apace 

• Entered obatarle area 

• Height greater than 52 Inchea 

• Incremental height greater than 6 Inchea 

Arm coordlnatea 

• Exceeded allowable range 

• Elbow torque greater thnn 50 foot-poundx 

• Wriat Increment greater than 90 

Hand coordlnatea 

• Exceaa hand force 

• Ci Ip greater than zero 

• Squeeze lera than 10 pounda 

a Reach Increment greater than 5 Inchea 

• Lift greater than 15 pounda 

Individual aenaora 

• Any touch aenaor on 

• Right fingertip force greater than 0.1 

ounce 

• Both Jaw forcea greater than 1 pound | 

Elapned time 

• Time greater than preaet 1 wit 


Notw • Aaterlaka denote emergency criteria, 

and bulleta denote operational criteria. 


MEANS 01 MEASURING THE CRITICAL FORCES 

To carry out the above manipulation taaka, vari- 
ous contacts with and pushes against objects In the 
environment must be sensed. Several methods of sens- 
ing these forces using manipulators are described In 
the following paragraphs. 

Joint Forces 

The force or torque at each Joint In the manip- 
ulator can be sensed by measuring either the motor 
current In electric systems or the back pressure In 
hydraulic systems. This -s particularly easy In 
electrically driven manipulators because the torque 
motor Itself Is used as the sensor, thus requiring no 
additional sensors. 

The use of Joint forces as measures of contact 
between the object and the end effector Is limited by 
several factors. Joint forces are contaminated by 
the weight of both the manipulator segments and the 
load. In addition, when the arm is in motion, chang- 
ing acceleration forces, changi ng centripetal forces, 
and reaction forces developed due to motions In other 
Joints, all further contribute to the )olnt force 
contamtnat Ion. 

Joint lorce measurements are also 11- d by the 
back-drive friction of the Individual Jo nts. 


Depending on the gearing, more th..n 10 perrent of the 
force exerted by a given Joint la likely to be required 
to back-drive that Joint. Though capable of driving 10 
pounda, such a Joint could sense only I pound. A force 
applied to the hand may back-drive some Joints <the 
freest ones) but not others, thus giving false informa- 
tion concerning the applied force vector. 

In spite of these limitations, Goto (1972) has 
used Joint forces to pack blocks tightly on a pullet. 
Innue (1971) compensated Joint forces for gravity load- 
ing by sieasurlng and storing static Joint forces before 
tusk Initiation, Using changes In the Joint force, he 
programmed a manipulator to Insert a pin Into a hole 
and to turn a crank. Considerably refining the tech- 
nique, Paul (1972) compensated Joint forres for boih 
gravity and acceleration loading and demonstrated sev- 
eral placing and sliding tasks. Another use of Joint 
forcea la the detection of colllalons against an ob- 
stacle, Restricting the use of Joint forres to the 
range from 30 to IOO perrent of a Joint's maximum force 
capability should avoid many of the complexities of 
compensation and back-drive limitations. 

Separate Sensing Couple 

Another means of measuring con. art between the end 
effector and the environment Is to measure the force 
couple at some point on the manipulator. The force 
couple consists of a torque vector and a force vector. 
Together, these forces completely describe the reaction 
force c t the point where the manipulator la rut. The 
obvious place to make this measurement Is between the 
end effector and the last Joint of the arm, as sug- 
gested by Sehelnmnnn (1969). Here the sensing Is In 
close proximity to tne load and, because the factors 
Influencing the signals from external contact are due 
only to the gravity and acceleration loading from the 
combined hand-ob)ect mass, the sources of contamination 
are significantly reduced. 

Thus, In moving from the Joints to the wrist, the 
sensing problem becomes greatly simplified. The major 
portion of the weight and the varying geometry nrc 
both removed from the sensing scheme. Assuming the 
weight of the end effector to be one-tenth the weight 
of the arm, wrist sensing rather than Joint sensing 
expands the ..'aeful force range by a factor of 10, al- 
lowing smaller forces to be measured. A wrist sensor 
for computer control of an arm was used by Groome 
(1972) to permit sliding a pin In a closely toleranced 
hole and aligning the wrist to a flat object. 

Touch Sensing 

The most sensitive and direct method ol measuring 
contact between an object and the ■ nd effector is to 
mount sensors on the outer surface i of the end effec- 
tor. Such sensing plates can have a mass of only a 
few grams, ami they In no way reduce the magnitude of 
the forces applied to the arm. With such a low mass. 

It Is not necessary to compensate for either gravity 
or arm acceleration, and forces on the order of grams 
can be sensed directly. I'ncompensated touch sensors 
are easily 1000 and 100 times, respectively, more 

















FIGURE 6 SIX-AXIS WRIST SENSOR GEOMETRY 


photot ransi ator light paths, wh.ch are broken by pins 
attached to the hand yoke. The motion of these pins 
will change the position of the shadow cast upon the 
square 1 i gh t-*ensi t i ve area of the phototransistors 
by the edge of the pin. Electrical s tennis cor respon- 
ding to the three forces and three torques aie ob- 
tained directly by subtracting the two constituent 
photocurrents . 

A useful feature of this system is that the 
weight of the hand drive motor balances the weight of 
the Jaws, as shown in Figure 7. Thus, the torques 
measured at the wrist sensor do not reflect hand 
weight. Proper balancing permits manipulation with 
lighter loads. This is similar to the mathematical 
compensation previously described, except that it is 
done prior to sensing and hence does not require such 
highly linear sensors. 

Touch Sensors 


The seven external sensing plates that cover each 
law activate proportional sensing elements. Those 
plates are uniformly sensitive to force over their 
surface and deflect approximately 1 mm under load con- 
ditions. Since the sensors were incorporated directly 
into the Jaw, they are very rugged. Because of the 
experimental nature of the hand, the external sensing 
plates seen in Figure 5 were designed to be replace- 
able and can be constructed of hnrd rubber or metal. 
The force range for each sensor depends upon a com- 
pliant element that can be ensily changed to vary the 
full scale sensitivity from 5 g to 5 kg Since the 
sensors are linear over a 100- to -1 range of forte. 
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FIGURE 7 WRIST INTERIOR SHOWING HOW WEIGHT OF 
SERVOMOTOR BALANCES WEIGHT OF HAND 

a simile sensitivity esn he useil tor different 'asks. 
The addition of composite or nonlinear eompllart ele- 
ments will permit the force ratine to he expan ted 
greatly . 

Intenrul to the inside surface of each Jaw Is a 
3 t 6 matrix of sc'islrg buttons, each with the same 
properties as the external sensing plates, as shown 
in Figure 8. With this array of sensors, it is pos- 
sible to derive simply control signals that will per- 
mit turn, ' '1st, anti sweep during Jaw c'oaure to be 
governed by the contours of the object, as previously 
shown in Figure 3. The tactile information t rom the 
)aw sensor matrix can lie used to find the location of 
objects in the Jaws and to compensate for faulty po- 
sitioning by motions in reach and lift. 


ooo 

ooo 

ooo 

ooo 

ooo 

000 

1 r 

FIGURE 8 THREE BY SIX SENSOR ARRANGEMENT ON 
INSIDE SURFACES OF JAWS (FULL SIZE I 







SENSOR PAD 

FORCE [ BOD y Qf tongs 


Finally, the base of the law contain* <1aop 
notcha* for attachment of fool* itlrartly to tha art at. 
A awltch In tha arlat Indicates that tha T-handle I* 
ft.-mly *aatad and that tha tool can he grasped. Tha 
tnalda Jaw aenaor* signal alien proper graap haa Iteen 
achieved and tha tool firmly grasped. Then, ualna the 
wriat aenaor, force* on the tool can be detected, and 
further aenaor-cont rol led miinlputut ton* can be per* 
formed . 


COMPLIANT WASHER 


The configuration of the touch aenaor* elthlr one 
law I* ahown In the croa* aectlon of Figure 9, Ttana* 
ductlon from external force to electrical aign.il oc- 
cur* In teo atagea. Flrat, a compliant washer In each 
aenaor determine* the deflection of a light vnne from 
tha external force. Then, the vane control* the light 
falling on a phototranalator, a* ahown In Figure* 10 
and It . 
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FIGURE 10 CROSS SECTION OF SENSOR WITH 
COMPACT SHUTTER 
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FIGURE II EXPLODED DRAWING OF SHUTTER 


BOTTOM SENSE PLATE 
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FIGURE 9 CROSS SECTION THROUGH JAW 


Paul, II. Modelling, Trajectory Calculation and 

Servolng of a Computer Controlled Irm. Stanford 
Artificial Intelligence Project Memo AIM-177, 
Stanford University, Stanford, California, 
November , 1972 . 


MII-1 , A Computer Operated Mechanical 
In 1962 Spring loint Computer Conference 


Schelnmnn, A. D, llealgn of a Computer Controlled 

Manipulator. Stanford Artificial Intelligence 
Project Memo AIM-92, Stanford 1'nlverslty, Stan 
ford, California, June 1969 


AF'IPS Conference Proceedlnga, A'ol. 21, pp. 39 


51 , Nat Iona I I’rean 


T. Compact Packaging by Robot with Tactile 
Sensor* Proceedings "I I lie second I n I e rnatt on a 1 


Seirmerl lng , W Robots are Herr*. Assembly Fngineerl 
Aol. 15, p. 42-49 , April 1972' 


alum on Industrial Robot*. I IT Hesearch 


Institute, Chicago, Illinois, 1972 


Whitney, I). E. Resolved Rate Control of Manipulators 
and Human Prostheses IEEE Transactions on Man - 
Machlne Systems. A'ol. AIMS-10, p 47-53, June 


Groome, R, C. . Jr. Force Feedback Steering of a Tele 
operator System. MIT Charles Stark Draper laib. 
Report T-575, Cambridge, Massachusetts, August 
1972. 



